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Abstract. Open-ended problem-solving 
skills have been identified as employable 
skills, which undergraduate students 

will need to have in order to cope with 
real-life problems in their workplaces after 
graduation. However, the development 
and assessment of these skills have been 
inconsistent across courses offered in 
universities. This research examined under- 
graduate students’ ability to engage with 
open-ended chemistry problems, and the 
influence of their working memory space on 
problem solving. The research is a descrip- 
tive research design that employed a survey 
method for collection data. A purposive 
sampling technique was used to recruit 

665 students from 19 chemistry depart- 
ments in Nigerian universities. The students’ 
ability to solve open-ended problems was 
measured in terms of their performance 
and was found to be below average (< 50). 
The female students outperformed their 
male counterparts in the chemistry open- 
ended problem-solving test (ChemOPST). 

A strong positive correlation was found to 
exist between students’ working memory 
capacity and their performance in the Che- 
MOPST. The research concludes on the need 
for experiential-based learning of chemical 
concepts which could help students build a 
knowledge base that is required to engage 
with problems that have real-life applica- 
tions. 
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Introduction 


Problem-solving skill is one of the desirable attributes required by univer- 
sity graduates for employability in professional or occupational environments 
(Overton & Potter, 2011; Sarkar, Overton, Thompson, & Rayner, 2016). The 
concepts of graduateness and employability have been described in relation 
to intellectual or higher-order cognitive development that is characterized by 
question asking, critical reasoning, decision making, system/lateral thinking 
and problem solving (Overton & Potter, 2008; Zoller, 2012; Zoller & Pushkin, 
2007). The changing professional standards, new workplace demands, and 
changes in learning theory have informed a plethora of research in problem 
solving and the consequent revision of science curricula with an intention to 
create learning environments that would provide learners with opportunities 
to develop higher-order cognitive skills (HOCS) through problem solving (e.g., 
Cardellini, 2006). Problem solving is important and plays a vital role in science 
curricula and classroom practice (Lorenzo, 2005). 

Wood (2006) defined problem as“a situation where at present the answer 
or goal is not known’ (p. 98). Generally, problems are of two types: Closed-ended 
and open-ended. Closed-ended problems are the types with only one correct 
answer, especially when it requires the application of algorithms to arrive at 
an answer. These types of problems come in the form of in-chapter or as end- 
of-chapters questions in textbooks. They serve the purpose of reinforcing 
concepts already learnt in the corresponding chapter. Open-ended problems 
are those which have many solutions or no specific solutions for the problem 
as defined. These are the problems Overton and Potter (2011) described as 
characteristic of problems encountered in real-life contexts, workplace, and 
academic research. 

The type of problems that are largely encountered within the classroom 
settings bear the semblance of close-ended problems. While the goal may not 
immediately appear obvious, the way and manner the problem is structured 
presumes that a goal has been specified. Such problems begin with, ‘find the 
volume of’; ‘how many moles of...?’; ‘show that... Wood (2006) explained that 
problems of this type could become routines or practice questions. With such 
questions, students who are skilled in algorithmic manipulation can substitute 
one set of data for another, and a solution is immediately reached. Johnstone 
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(2006) suggested that for a student who can recollect the strategy to tackle such a problem, there remains no problem. 

However, contextual or real-life problems take different forms that are not open to algorithmic manipulation; 
but may require some level of lateral or divergent thinking to reach solutions (Wood, 2006). Overton and Potter (201 1) 
affirmed that criteria for success in such problems are rather different from the closed-ended problems. Lamentably, 
the experience is that teachers and students at the different levels of education are so used to working towards ‘the 
correct answer’ in problem-solving activities. The consequences of this, create erroneous impression in students to 
think that for every problem in science; there will always be a single correct answer. Wood (2006) firmly cautioned 
against the risk of cultivating in students that “all is known” about science-as if it is a discipline to which students 
cannot make their individual contributions. Oftentimes, this inaccurate conception of a single correct answer is per- 
petuated across the different levels of education. Stating the obvious, most of the assessments (examinations) at all 
levels of education reinforce this incorrect nature of science (Bennett, 2004). 

For a better understanding of the nature of problems, Johnstone (1993) categorized problems based on three 
variables that are linked with problem solving. These include the: available data; methods; and the intended outcomes. 
Furthermore, Johnstone (1993) identified eight (8) problem types based on the amount of information available on 
each of the variables (see Table 1). 


Table 1 
Types of problem 
Type Data Methods Outcomes Skills required 
1 Given Familiar Given Algorithmic manipulations 
2 Given Unfamiliar Given Look out for, and compare alternatives to known methods 
a Incomplete Familiar Given evaluate the problem to determine other data that are required 
4 Incomplete Unfamiliar Given consider possible methods and made decision on the data required 
5 Given Familiar Open Make appropriate decisions about the outcomes/goals. 
Explore other networks of knowledge 
6 Guen eee Open Make appropriate decisions about the goals and the methods to apply 
Explore knowledge networks 
7 Incomplete Familiar Open Specify the goals, and determine what data are seen to be incomplete 
3 incomplete agarsilier Open Suggest the goals and appropriate methods to arrive at the solution. Think of 


the need for additional data. All the above skills. 
Source. Adapted from A. H. Johnstone (1993). Creative Problem Solving in Chemistry 


Problem types 1 and 2 are the problems that students often encounter in their learning of chemistry (Reid & 
Yang, 2002). They are problems found in textbooks as end-of-chapter questions and examination papers. Johnstone 
(1993) described them as exercises that are algorithmic in nature. Types 3 and 4 are considered to be more difficult and 
require seeking data/reasoning to reach a solution. While types 5 to 7 are more open problems, problem type 8 which 
has incomplete data, and requires unfamiliar methods with ill-defined goals is typical of an open-ended problem. 

Problem solving in chemistry combines the features of problems found in mathematics and physics and adds 
its own distinct chemical characteristics in quantitative chemistry topics such as stoichiometry, chemical reaction 
and synthesis, and analysis (Cardellini, 2006). Chemistry examination contains a variety of question types. While 
some questions measure knowledge and recall of chemical facts; others assess students’ ability to carry out simple 
calculations, interpret data, write short explanations or design and report experiments. Regardless of whether the 
questions require lower/higher-order cognitive skills (L/HOCS), one general characteristic of the prevalent chemistry 
examination is the close-ended nature of the questions/problems. Unlike closed questions, open-ended problems 
are relatively new, and the potentials are yet to be fully explored (Scottish Qualification Authority, 2010). 

Open-ended problems present challenges that require students to demonstrate their understanding of chemical 
processes. Reid (2000) noted that when students encounter problems, it is then that they are at the threshold of learn- 
ing. If this proposition is anything to go by, it is important to provide students with firsthand experience on problem 
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solving that have real-life applications. Rewarding classroom moments could occur when students are challenged 
to take ownership of their learning. This active engagement energizes students to take their understanding to an in- 
creasingly higher level (Belt, Evans, McCreedy, Overton, & Summerfield, 2002; Overton, Potter, Leng, 2013; Reid, 2000). 

The issue of gender differences in problem-solving related research has precipitated a variety of research over 
time and will no doubt, continue to a strand of research in science education (Al-Ahmadi, 2008; Hindal, 2007; Hindal, 
Reid, & Whitehead, 2013a; Pomerantz, Altermatt, & Saxon, 2002). In spite of the extensive research on gender differ- 
ences in mathematics and science, results on students’ performance based on their gender has been inconsistent 
(Beller & Gafni, 2000; Hindal et al., 2013a; Overton et al., 2013). For instance, Beller and Gafni (2000) compared gender 
differences in open-ended and multiple-choice questions in mathematics tests over a period of two years and reported 
that boys outperformed the girls. While the findings indicate inconsistent patterns with respect to gender effects and 
item-type, the researchers posited that item-type alone would be inadequate to make conclusions on gender differ- 
ences in the mathematics tests. This submission created a research gap for the investigation of cognitive variable(s) 
associated with high performance in cognitively demanding tasks that are algorithmic and open-ended in nature. 

Hindal et al. (2013a) explored how students’ performance differs in an examination and a variety of tests that are 
related to learners’ characteristics (e.g., field dependence/independence, convergence/divergence and visual-spatial 
abilities), as well as working memory capacity. They found out that female students were superior in all the measurements 
compared to their male counterparts, except for working memory capacity, where there was no significant difference. 

Overton et al. (2013) investigated the different methods that students use in solving rich-context open-ended 
problems in chemistry and their cognitive abilities, gender, and cultural background, but no result was reported to 
explain students’ performance based on gender. It is therefore, a focus in this research to contribute to the literature 
by exploring the limiting capacity imposed by the working memory space and the explanations it offers for the indi- 
vidual student to engage with open-ended problems considering their gender. 


Theoretical Framework 


This research adopted one of the most widely accepted theoretical models—the information processing model 
by Baddeley and colleagues to describe the function of working memory space that temporarily holds and processes 
information in relation to problem solving (e.g., Baddeley, 1986, 1999, 2002; Baddeley & Hitch, 1974). The model fits 
well within the empirical basis for this research and therefore, guides the study in exploring how students’ working 
memory capacity can influence their success or performance in solving open-ended chemistry problems. 

Baddeley and Hitch (1974) expanded on the initial multi-store model by Atkinson and Shiffrin’s (1968) and de- 
veloped an alternative model of short-term memory (STM), which they called working memory. The working memory 
(WM) is the part of human brain that receives and temporarily stores incoming information. It is also described as a 
mental or thinking-holding space that has a limited capacity (Danili & Reid, 2006). Hindal, Reid and Badgaish (2009) 
further described working memory as a space where intelligent thinking, information interpretation, understand- 
ing and problem solving occurs. While, it is known to have a limited capacity, Johnstone (1997) noted that working 
memory can have a strong influence on learning. 


Figure 1 
The information processing model adopted from Atkinson and Shiffrin (1968) 
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In Figure 1, the information processing model (IPM) suggests a mechanism and control for learning. Informa- 
tion/instruction can be admitted as perception, and it is controlled by what a student already knows. The perception 
filter is what students can use to filter, control noise and select relevant information. Information that is filtered 
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or selected goes through the perception filter. Thereafter, the message of information is accommodated into the 
students’ working memory space to allow for further processing. The processing of incoming information usually 
requires information from the long-term memory for interaction and sense making (Johnstone, 2006). In other 
words, if the processed material is to be stored, it looks for previous knowledge and understanding as linchpins in 
the long-term memory (LTM) to anchor the message/information as new knowledge (Bahar, Johnstone, & Hansell, 
1999). This implies that a large interrelated system of knowledge and experience is enriched, from which a learner 
can draw for effective learning. On the contrary, if sense is made from the processed information through faulty 
attachment, it gives rise to alternative conceptions that may not be easily debunked, especially in situations where 
a learner found such faulty knowledge sufficient. Johnstone added that, if the new information accommodated 
in the working memory space is not linked to the LTM and it becomes necessary to retain it, the new information 
may however, enter the LTM unattached as rote learning or mere memorization of facts. When such information 
is needed, a recall is usually difficult. This model unveils some difficulties students encounter and why effective 
learning may not take place. 

First, the IPM reveals that the working memory space of an individual is limited and has a fixed capacity. It 
can only handle limited information within a particular time (Baddeley, 1986; Johnstone, 1997). In addition, previ- 
ous research has further confirmed that the working memory space is genetically fixed and limited and has been 
reported that adults generally have a capacity of accommodating 7 + 2 units of information (Gathercole, Lamont, 
Alloway, 2006; Reid, 2009). The working memory space of a student can be measured using some psychological 
instruments. In literature, the digit-span backward test by Weschler (1997) and figural intersection test (FIT) by 
Pascual-Leone (1970) have been used to measure students’ working memory capacity (Johnstone & El-Banna, 1986). 
According to Weschler (1955), the amount of information that can be accommodated in the working memory space 
ranges from four to seven. Consequently, the pieces of information or steps required to solve a problem can be a 
determining factor to successfully solve problems. Johnstone (2006) noted that in a situation where students do 
not have the mastery of algorithmic steps to tackle the demand of a problem, they try to manipulate too much 
information beyond their working memory capacity, and as a result, effective learning is inhibited. This is because 
conceptual overload has occurred. The resultant effects often manifest in the form of lack of motivation and nega- 
tive attitude learners’ display towards chemistry as they may have shut down on the basis that chemistry requires 
a lot of information processing with limited working memory space. 

Extensive research on cognitive psychology provides further insight into how the STM, alternatively referred 
to as working memory operates in relation to holding and processing of information (Baddeley, 1986; Engle & 
Kane, 2004). Specifically, the working memory capacity has been documented as one of the important learners’ 
characteristics that accounts for high academic performance (Hindal et al., 2013a; Hunt, 1999). Nalliah (2012) 
noted that in disciplines such as mathematics, chemistry and medicine that involves domain-specific learning, 
the skills and principles of working memory have been adopted as fundamental to experience-based learning 
(Noble, Miller, & Heckman, 2008). 

The working memory space can be reduced to provide space to hold and process information. Researchers 
suggested the use of schema or chunking of information (e.g., Johnstone, 2006; Reid, 2009). Anderson (1977) 
describe schema as a mental framework that can be used to organize and process information. For problem solv- 
ing, students can use schema to break down complex problems into smaller pieces, and then develop a familiar 
problem-solving procedure to reach an answer. The use of schema as a strategy suggests organizing the working 
memory for efficient use (Miller, 1956). According to Miller (1956), chunking allows the breaking down of a large 
amount of information into smaller pieces/chunks to reduce cognitive overload in the problem solvers. Once cogni- 
tive overload occurs because of the large amount of data in the working memory space to be processed, problem 
solving may be inhibited. For students to become successful with open-ended problems, the use of schema and 
chunking could be good strategies to employ during problem solving. Students’ ability to engage successfully with 
open-ended chemistry problems, putting in perspective the limitations of the working memory space, will inform 
our interpretations of the results from this research. 


Literature Review 
One of the dominant research strands in science education focuses on assessment that can help students 


develop HOCS through question asking, evaluative and lateral thinking, and problem solving (Barak & Dori, 2005; 
Zoller, 1993, 1999, 2012). However, the types of assessments that are prevalent in typical traditional chemistry 
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classrooms are the conventional algorithmic-based questions that promote LOCS (Bennett, 2004; Johnstone, 1997; 
Zoller &Tsaparlis, 1997). Usually, these questions have unique correct answers (Surif, brahim, & Dalim, 2014). While 
such problems certainly provide students with necessary practice to apply the fundamental course concepts, their 
exclusive use could make students erroneously believe that all chemistry and real-life problems are similarly struc- 
tured. Reid and Yang (2002) stated that important real-life problems are most times imprecise and open-ended in 
nature. Hence, the implication of continuously presenting undergraduate students with well-defined or single-right 
answer questions/problems could leave them unprepared for the more open-ended problems they are likely to 
come by in real-life contexts and the industry after graduation. 

The current economic situation around the world demands that graduates are equipped with academic and 
special skills required to navigate in fast-changing professional environments and world of work (Overton & Potter, 
2011). Similarly, employers have also placed demand on institutions of higher learning to produce graduates who 
are: equipped to solve new or novel problems; have good communication skills; team players; handle complex 
data; and have personal skills (Coldstream, 1997; Duch, Groh, & Allen, 2001; Lowden, Hall, Elliot, & Lewin, 2011; 
Mason, 1998; Zoller & Tsaparlis, 1997). This becomes necessary when Hanson and Overton (2010) reported that, 
new graduates in their first two years of post-university role, acknowledged that while their education in terms of 
chemistry learning has been all-inclusive, they felt inadequately prepared in terms of transferable skills (problem 
solving, critical reasoning, evaluative thinking, decision making, etc.) needed in order to cope at their workplaces 
(Sarkar et al., 2016). The possibility of engaging students with open-ended problems in problem-based learning has 
been confirmed to hold potentials for developing an inclusive range of these 21st century learning skills (Duch et 
al., 2001; Hanson & Overton, 2010). The nature of open-ended problems is such that the data given is incomplete, 
goals are ill-defined, and it requires the use of unfamiliar methods (Johnstone, 1993; St. Clair-Thompson, Overton, 
& Bugler, 2012). Open-ended problems require the application of HOCS to arrive at a solution. While there are 
studies that have reported successful implementations of how students solve chemistry open-ended problems 
as group work (Belt & Overton, 2007; Grant et al., 2006; Summerfield, Overton, & Belt, 2003; Reid & Yang, 2002), 
research that has explored individual student’s ability to engage with open-ended problem solving are relatively 
few (e.g., Overton & Potter, 2011; Overton et al., 2013; St. Clair-Thompson et al., 2012). 

Recent studies have focused on students’ comparative performance in algorithmic, open-ended problems 
(Overton & Potter, 2011; Surif et al., 2014), and in relation to cognitive variables that are fundamental to successfully 
engage with open-ended chemistry problems (St. Clair-Thompson et al., 2012). Surif et al. (2014) have compared 
students’ performance in algorithmic, conceptual, and open-ended chemistry problems but the cognitive variable(s) 
that mediate successful problem-solving were not examined. The findings showed that majority of the students 
were unable to solve or tackle the conceptual and open-ended problems. In another study, Overton and Potter 
(2008) reported that students’ cognitive styles correlated with their success in problem-solving. Specifically, there 
was a relationship between field independent students and their open-ended problem-solving ability. With a small 
sample size, the result of their study cannot be generalized. St. Clair-Thompson et al. (2012) compared students’ 
scores in algorithmic and open-ended chemistry problems and noted some discrepancies. In their study, students’ 
scores in both tests were not significantly linked, and this suggests that students had difficulties with open-ended 
problems characterized by HOCS (Overton & Potter, 2011). 

Research has documented a few cognitive variables (field-dependence/independence, convergent/divergent 
characteristic, and working memory) that are pertinent to problem solving and also facilitate the achievement of the 
problem solvers (Solaz-Portolés, & Sanjosé, 2007). This further suggests the existence of a link between the cogni- 
tive variables and problem solving. While there are studies that have reported on working memory as a predictor 
of algorithmic problem solving (Al-Ahmadi & Oraif, 2009; Ali & Reid, 2012; Danili & Reid, 2004; Overton & Potter, 
2011; Reid, 2009; Tsaparlis, 2005), others indicated no significant relationship between students’ working memory 
and their ability to solve open-ended problems (e.g., Overton & Potter, 2011). Contrariwise, St. Clair-Thompson et al. 
(2012) reported that open-ended problem solving partly rely on working memory capacity but noted that a large 
amount of variance on performance could not be unaccounted for. Since there are inconsistent results in litera- 
ture, the present research used a baseline assessment to measure students’ ability to solve open-ended chemistry 
problems. Furthermore, it was predicted that students’ working memory capacity would influence their ability to 
solve open-ended chemistry problems. In this research, “ability” is considered as students’ high performance in 
solving open-ended chemistry problems (Hindal, Reid, & Whitehead, 2013b). In clear terms, undergraduates who 
score high in the ChemOPST will be described as being able to solve the open-ended chemistry problems. 

The overall aim of this research was to determine students’ ability, and the influence of their working memory 
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Capacity to solve open-ended chemistry problems. To guide the present research, the following research questions 
were raised to address the main purpose of the research: 
1. What is the performance of undergraduate students in open-ended chemistry problems? 
2. Dothe male students score higher than their female counterparts in the ChemOPST? 
3. Dothe male students score higher than their female counterparts in the FIT? 
4. ls there any relationship between undergraduate chemistry students’ working memory capacity and 
their ability to solve open-ended chemistry problems? 


Research Context 


In Nigeria, a Decree of the erstwhile Federal Military Government (amended as Decree No. 48 in 1988) estab- 
lished and empowered the national universities commission (NUC) to prescribe academic standards for all courses 
offered within the Nigerian universities. As a collaborative endeavor, the Commission in partnership with academic 
staff of the universities developed what was earlier referred to as the minimum academic standards (MAS) for all the 
courses in 1989 (NUC, 2007). The document was immediately approved for used in Nigerian universities in 1989. 

The MAS documents became major tools to guide the conduct of accreditation exercises of academic pro- 
grammes offered by Nigerian universities. In 2001, the NUC began the process of revising the MAS documents 
after a decade of implementation. The review became important as “frontiers of knowledge” in all the science 
disciplines have advanced in the light of new information generated through research. In addition, the revision 
was needed to update the documents in terms of integrating entrepreneurship, peace and conflict studies that 
will equip university graduates with relevant skills and competencies; and to enhance their capability to contribute 
meaningfully towards the socio-economic growth of the country and global competitiveness. 

In 2007, the Commission merged the benchmark standards and the reviewed MAS into new documents, 
which is now referred to as the benchmark minimum academic standards (BMAS) for the different courses. The 
BMAS clearly articulates the learning goals/outcomes and skills anticipated from undergraduates at the end of 
each academic year of their disciplines. The document is not overly prescriptive, but rather flexible and creates 
room for universities to incorporate emerging course contents that are consistent with international best practice. 

In principle, the training of scientists is designed to be thorough, creative, innovative and relevant to gradu- 
ate employment opportunities. The undergraduate chemistry programme is designed to promote an appreciable 
knowledge of the centrality of chemical science to human lives, as well as its cross-cutting relevance to other 
branches of science. The students are also expected to acquire competencies that can be applied in chemical and 
non-chemical areas; confidence for employment; and fundamental chemical knowledge and skills required to 
pursue further studies in chemistry-related fields and academic research (NUC, 2007). 

In addition, chemistry graduates should be able to apply the knowledge and skills they have acquired to solve 
academic and real-life practical problems that have relevance for allied industries towards solving societal and 
national needs (NUC, 2007). However, the development and assessment of these abilities and skills has not been 
consistent in several courses/programmes offered in Nigeria. Research in open-ended problem-solving and work- 
ing memory is still in its infancy and not widespread within the context of the study. Therefore, this research aimed 
at using baseline assessment to measure undergraduate students’ competence to solve open-ended chemistry 
problems, and to determine the influence of their working memory capacity in problem solving. 


Research Methodology 
Research Sample 


This research was a descriptive research that employed a survey method to measure undergraduate students’ 
ability to solve open-ended chemistry problems, and the influence of their working memory capacity. The sample 
population was purposively selected based on two criteria. One, the universities must have a minimum of 5-year 
experience in graduating students from the chemistry department; and two, it must be NUC accredited universities 
with a well-established undergraduate chemistry programme that aligns with NUC benchmarking guidelines. The 
initial sample of participants enlisted for the research were 830 undergraduate chemistry students drawn from 
across 19 universities in Nigeria. However, due to voluntary withdrawal and incomplete response to the tests, the 
final sample consisted of 665 students. The frequency and percentage distribution of participants by gender (i.e. 
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female and male) were 351 (47%) and 314 (53%) respectively. Since, none of the students indicated a different 
gender identity from their sex at birth, gender is operationally defined as male and female in this research. 


Research Instruments 


The research instruments consisted of a pre-attitude questionnaire, a FIT, the ChemOPST and a post-attitude 
questionnaire. The pre-and post-test attitude questionnaire used bipolar statements to measure students’ attitude 
toward chemistry and to solving open-ended chemistry problems. The results on students’ attitude before and 
after the administration of ChemOPST will be reported in another article. 

In order to measure students’ working memory space, the FIT developed by Pascual-Leone (1970) was adopted 
and used. In the FIT there are two groups of simple geometric shapes. On one side is the FIT with overlapping shapes 
that have an intersect inside all the shapes, and on the other side are the same shapes but separated from each 
other (see example in Figure 2). Altogether, there were 30 overlapping shapes with a common intersect in each 
item of the FIT. The number of geometric shapes ranges from 2 to 8. The FIT is usually timed, and the intersect of 
each group of shapes taken as an item must be located and shaded within 20 seconds. The validity and reliability 
of the FIT has been demonstrated by Johnstone and El-Banna (1986, 1989). Assessing undergraduate students’ 
working memory capacity using the FIT allows to draw relationship between students’ performance in FIT and 
the ChemOPST. Research has shown that scores on working memory task can be a useful predictor of success in 
cognitively demanding task in science and mathematics (Baddeley, 2002; St. Clair-Thompson & Gathercole, 2006). 


Figure 2 
An example of a figural intersection test 





The validity of the ChemOPST was established using several strategies. To ensure face and content validity of 
the ChemOPST, experts in teaching chemistry assisted to review the questions and provided feedback. The instru- 
ment was given to two professors; a science educator and a chemist. They were requested to provide comments 
on the content to be assessed, which were discussed between the authors and incorporated into the ChemOPST. 
In addition, the ChemOPST was administered in paper and pen format to chemistry teachers and postgraduate 
students enrolled for chemistry programme at master’s level. They were requested to take the test, as well as pro- 
vide their feedback with respect to the chemistry content assessed, the suitability of their answers, and the scope 
of chemistry content covered. The comments made, especially those from postgraduate students who registered 
for chemistry programme, were considered in fine-tuning the ChemOPST. 

For further validity, the ChemOPST was also administered to 30 students at a different university who were 
not part of the final sample for the research. They were requested to verify the clarity of problems, appropriateness 
of language and to also suggest suitable duration or period for the test such that time will not be a constraint for 
the test takers. The reliability of the ChemOPST was further subjected to a test-retest method within a period of 
three-week interval. Scores obtained from the first and second administrations of the instrument were correlated 
using Pearson-product moment correlation statistic to obtain reliability indices for the ChemOPST. A calculated 
reliability coefficient of 0.76 was obtained and considered adequate for the research. 


Data Collection 


Having secured the approval of the Faculty of Education Ethical Clearance Committee to conduct this re- 
search that involves human subjects, the researcher and research assistants visited participating universities to 
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obtain permission to enrol students from chemistry department to participate in the research. The students were 
requested to indicate their willingness and to fill the consent forms to participate in the research. Thereafter, the 
research instruments were administered in each of the selected universities with the consent, cooperation and 
assistance of the chemistry lecturers in those universities. The students were given 1 hour 30 minutes to attempt 
all the problems and were instructed at the beginning to write down all their thinking in the booklet provided. 
They could seek clarification if they so wished, but only on the instructions. 

Open-ended problems are quite difficult to score compared to close-ended problems. This is because, the 
problem-solving process or strategy is equally as the outcome. In accordance with established practices in this 
area, marks were distributed for identifying the goals, the quality of the approach and for the assumptions and 
estimations made (Overton & Potter, 2011). Each of the open-ended problems was allocated 10 marks. The marks 
were distributed accordingly to match the designed marking scheme (see supplementary material for samples of 
open-ended problems and making guides). Only about 665 scripts for each of the FIT and ChemOPST were retrieved 
and scored. After the test scripts were scored and recorded in MS Word Excel, they were exported into the window 
version of statistical package for social sciences (SPSS) software 25.0 for analysis. 


Results of Research 


The results of the research are presented here, in the order in which research questions were raised to guide 
the study. Research question 1 focused on students’ ability to solve open-ended chemistry problems. There were 
665 students who consented and participated fully in the chemistry open-ended problem-solving test (ChemOPST). 
Figure 3 shows the students’scores obtained from the ChemOPST. The mean score for the ChemOPST was 45.68 (SD= 
12.05). An examination of the shape of the histogram provides information about the distribution of students’ scores. 


Figure 3 
Mean score of students in the ChemOPST 


Chem OPST Score 


Mean = 45.65 
Std. Dey. = 12.047 
iM = 665 


Frequency 





Chem OPST Score 


Do the male students score higher than their female counterparts in the ChemOPST? 

Alternative hypothesis (H,): The scores of male students in the ChemOPST are higher than their female coun- 
terparts. 

In determining whether the male students score higher than their female counterparts in the ChemOPST, we 
conducted a Wilcoxon Signed Rank Test, the result of the test in Table 2 revealed a statistically significant difference 
in the scores of male and female students in the ChemOPST, Z = — 3.476, p < .001, with a small effect size (0 = 0.13). 
The median score of the female students (Md = 50.00) was higher than the male students (Md = 44.00). Therefore, 
the alternative hypothesis was rejected. 
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Table 2 
Wilcoxon signed rank test on students’ score in the ChemOPST by gender 
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Gender N Mean Rank 
Female 314 360.33 
ChemOPST Male 351 308.55 
Total 665 
Test Statistics? 
Mann-Whitney U 
Wilcoxon W 
Z 
Asymp. Sig. (2-tailed) 
Exact Sig. [2*(1-tailed Sig.)] 
a. Grouping Variable: Gender 
Median Score of the ChemOPST by Gender 
ChemOPST 
Gender N 
Female 314 
Male Oot 
Total 665 


ISSN 1648-3898 | print 
ISSN 2538-7 138 /oniines 


Sum of Ranks 


113145.00 
108300.00 
ChemOPST 
46524.00 
108300.00 
-3.476 
.001 
0005 
Median 
50.00 
44.00 
47.00 


The working memory capacity of the students was measured by the FIT. Figure 4 presents students’ scores 
obtained in the FIT. The mean score for the FIT was 17.14 (SD = 3.19) skewing fairly to the right. A closer look at the 
shape of the histogram provides information about the distribution of students’ scores. 


Figure 4 
Mean score of students in the FIT 


FIT Score 


Frequency 


FIT Score 





Mean =17.14 
Std. Dey. = 3.187 
HM = 665 
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Do the male students score higher than their female counterparts in the FIT? 

Alternative hypothesis (H,): The scores of male students in the FIT are higher than their female counterparts. 

Table 3 provides a Wilcoxon Signed Rank Test which indicates a statistically significant difference in the scores 
of male and female students in the FIT, Z = — 5.972, p = .000, with a small effect size (O = 0.23). The median score 
of the female students (Md = 18.00) was higher than the male students (Vd = 17.00). Therefore, the alternative 
hypothesis was rejected. 


Table 3 
Wilcoxon signed rank test on students’ score in the FIT by gender 


Gender N Mean Rank Sum of Ranks 
Female 314 379.71 119229.50 
PIT Male 301 291.21 102215.50 
Total 665 
Test Statistics? 
FIT 
Mann-Whitney U 40439.50 
Wilcoxon W 102215.50 
Z -5.972 
Asymp. Sig. (2-tailed) .000 
Exact Sig. [2*(1-tailed Sig.)] .000 


a. Grouping Variable: Gender 


Median Score of the FIT by Gender 


FIT 
Gender N Median 
Female 314 18.00 
Male O01 17.00 
Total 665 17.00 


The correlation between chemistry students’ scores in open-ended problems (measured by the ChemOPST) 
and their working memory capacity measured by the FIT was analysed using Pearson product-moment coefficient 
correlation. Preliminary investigations were conducted to ensure that there was no violation of the assumptions 
of normality, linearity and homoscedasticity. The results presented in Table 4 indicate a strong, positive correlation 
between the two variables, r= .81, n = 665, p < .001, with students’ working memory capacity been closely associ- 
ated with their ability to solve open-ended chemistry problems. 
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Table 4 


Correlation of students’ scores in the ChemOPST and FIT 


alas FIT Score 
Pearson Correlation 1 ove 
Sig. (2-tailed) .000 
ChemOPST Score Sum of Squares and Cross-products 96362.78 20551.47 
Covariance 145.13 30.95 
N 665 665 
Pearson Correlation Ol” 1 
Sig. (2-tailed) .000 
FIT Score Sum of Squares and Cross-products 20551.49 6745.27 
Covariance 30.95 10.16 
N 665 665 
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Note. **correlation is significant at the 0.01 (2-tailed) 


Discussion 


This research examined students’ ability to solve open-ended chemistry problems. Since working memory is 
known to play an important role in cognitive processes such as problem solving (Danili & Reid, 2006), the research 
also investigated the correlation between students’ working memory capacity (a cognitive variable) and their 
ability to solve the open-ended chemistry problems. Research findings revealed that students who started out 
to attempt some of quantitative problems by applying algorithms, soon realized that some of the problems were 
not opened to algorithmic manipulations. This is because the cognitive skills required to solve open-ended and 
algorithmic problems are different (Overton & Potter, 2011). The students’ ability was measured in terms of their 
performance in solving chemistry open-ended problems. It was found that majority of the students scored below 
50 marks in the ChemOPST. This result is expected, as this may probably be the first-time students are presented 
with questions that are open-ended. This assumption matches well with Bennett (2004) findings that examination 
questions from chemistry departments in United Kingdom and Australian universities were algorithmic in nature 
(Johnstone, 1993). 

The analysis of students’ score in the ChemOPST based on gender revealed a statistically significant difference 
with the female students performing better than their male counterparts in the ChemOPST. This result agrees with 
the findings of those of Hindal et al. (2013a) who explored how students differed in their performance in a range 
of tests related to learners’ cognitive variables based on gender. The female students were found to be superior to 
male students in all the measurement, except for the working memory capacity, where there was no significant 
difference related to gender. 

To determine students’ working memory capacity, the FIT designed by Pascual-Leone (1970) was administered 
to the students. As shown in Figure 3, the scores obtained from the FIT ranged from 7 to 26, with majority having a 
score on FIT ranging from 17 to 18. A comparison of students’ scores in FIT based on their gender revealed a statisti- 
cally significant difference between the male and female students at p = .0001 (1-tailed). The females performed 
better than the males in the FIT. This result probably suggests that female students were able to hold considerable 
amount of information and have more working memory space to process information, because they appeared to 
be quick at locating the intersect of geometric shapes within a short time and scoring higher when compared to 
their male counterparts in the FIT. 

The result of the relationship between undergraduates’ ability to solve open-ended problems, measured by 
ChemOPST and their working memory capacity demonstrated a strong, positive correlation between the two 
variables, r= .81, n = 665, p < .01. The implication of this is that students with relatively high scores in the FIT were 
the most successful in the open-ended problems. This result is consistent with the findings of (Overton & Potter, 
2008, 2011; St. Clair-Thompson et al., 2012). Overton and Potter (2011) examined students’ success in open-ended 
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chemistry problems that used real-life, rich context, and require the application of HOCS. A positive correlation 
exists between students’ working memory capacity when compared with their ability to solve algorithmic and 
open-ended problems, with a demonstration of a threshold effect. One important basis for this correlation could 
be that students who were the most successful in the ChemOPST adopted some more efficient retrieval systems 
without having to overload the working memory space, and thereby diminished the influence of working memory 
Capacity on their ability to solve open-ended problems. 


Conclusions 


Open-ended problem-solving ability is one desirable skill that is necessary for students to cope with the chal- 
lenges in new workplace after graduation. The question of whether undergraduate chemistry students possess the 
ability to solve chemistry problems that share similar characteristics with open-ended problems in real-life context 
was the focus of our study. The result shows that undergraduate students’ ability to solve chemistry open-ended 
problems was below average. While it is understandable that the assessments the students are familiar with are 
not largely open-ended in nature, we believe that exposing university students to this form of assessment early 
in their university education holds potentials to improving their problem-solving abilities. Therefore, the onus is 
on academics to employ methods of instruction and assessment that could help students develop higher-order 
cognition through decision making, system/lateral thinking and problem solving from the first year of their uni- 
versity education. This is important because many students are admitted into the university without these skills. 

One way that could help students develop ability in open-ended problem-solving may be the use of myriad of 
consistent and cumulative teaching-learning activities that involve a large portion of experience-based learning or 
problem-based learning to reduce overload of the students’WM capacity. It is equally important that modules and 
courseware materials to which students are being exposed, should reflect the predictions about learnings derived 
from the information processing model to improve students’ ability or performance in context-based learnings. 

Working memory capacity as a psychological factor showed relatedness or a positive correlation with students’ 
ability to solve open-ended chemistry problems as evident in this research. Assessment that requires the handling 
of information will tend to favour students with high working memory capacity. This implies that teaching, learning 
and assessment of undergraduate chemistry students must operate within the limitations imposed by the working 
memory. Therefore, for students to be able to cope with problem solving that requires a lot of information, they will 
need to reduce cognitive overload by making use of schema drawn from the long-term memory or by chunking 
pieces of information. University teachers could also help students avoid cognitive overload by preparing questions/ 
problems with little or no extraneous information/data that may not be relevant to reach the solution of the problems. 
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Appendix 

A sample of the Chemistry open-ended problem-solving test (ChemOPST). 

The Chewing Gum Problem 

People have chewed gum-like substances to freshen their breath and clean their teeth for centuries. Some 
people think that chewing gum does not only freshens the breath, but also cleans the teeth and helps to free 
them from decay. 

Let’s look at tooth decay. Does gum-chewing really help to keep teeth healthy? 

Here’s a list of what is in chewing gum: 


i. Chewing gum base: a synthetic rubber-like substance; 

ii. Sweeteners: sugar (sucrose) or sugar substitutes (like those used in diet coke); 
iii. Softeners: vegetable oil products like glycerine (glycerol); and 

iv. Flavourings: spearmint and peppermint oils. 


Discuss the possible answers to the problems below: 
1. What do you think is the difference between ordinary gum and “sugar-free” gum? 
2. After eating, bacteria will attack and break down carbohydrates like starch and sucrose (Sugarcane). Try to 
write down as much as you can, about the process in which carbohydrate is broken down in the mouth. 
3. We want to find out if the practice of using chewing gum helps to fight tooth decay or not. Discuss 
what information you need to reach an answer, based on your knowledge of the way carbohydrates 
are broken down. Make a list of what you need to know. 
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Marking Scheme for the ChemOPST 
Activity 


Identifying data 


Method 


Goals 
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Mark 
Separating relevant from irrelevant 1 
Making assumptions or estimations 1 
Applying known methods or strategies 2 
Developing new methods or strategies 2 
Defining the goal 1 
Working towards the goal 1 
Reaching the goal 1 
Checking the goal 1 
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